Three-dimensional measurement device having three-dimensional overview camera

ABSTRACT

A device for optically scanning and measuring an environment is provided. The device includes a first measurement device that emits a light beam in a direction to measure a distance to a remote target based at least in part on light reflected by the remote target. A three-dimensional camera coupled to a periphery of the first measurement device is configured to record an image of an object. A processor is operably coupled to the first measurement device and three-dimensional camera and is responsive to determine the three-dimensional coordinates of the measurement point based at least in part on the angles of rotation of the device and the distance. The processor further being responsive to determine the three-dimensional coordinates of a plurality of points on the object based at least in part on the angles of rotation of the device and the image.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application is a non-provisional application of U.S.Provisional Application 61/844,631 filed on Jul. 10, 2013, the contentsof which are incorporated by reference herein in their entirety.

BACKGROUND

The present disclosure relates to a coordinate measuring device. One setof coordinate measurement devices belongs to a class of instruments thatmeasure the three-dimensional (3D) coordinates of a point by sending alaser beam to the point, where it is intercepted by a retroreflectortarget. The instrument finds the coordinates of the point by measuringthe distance and the two angles to the target. The distance is measuredwith a distance-measuring device such as an absolute distance meter(ADM) or an interferometer. The angles are measured with anangle-measuring device such as an angular encoder. A gimbaledbeam-steering mechanism within the instrument directs the laser beam tothe point of interest. Examples of such a device include a lasertracker, a total station, and a time-of-flight (TOF) scanner.

Ordinarily the laser tracker sends a laser beam to a retroreflectortarget. A common type of retroreflector target is the sphericallymounted retroreflector (SMR), which comprises a cube-cornerretroreflector embedded within a metal sphere. The cube-cornerretroreflector comprises three mutually perpendicular mirrors. The apexof the cube corner, which is the common point of intersection of thethree mirrors, is located at the center of the sphere. It is commonpractice to place the spherical surface of the SMR in contact with anobject under test and then move the SMR over the surface being measured.Because of this placement of the cube corner within the sphere, theperpendicular distance from the apex of the cube corner to the surfaceof the object under test remains constant despite rotation of the SMR.Consequently, the 3D coordinates of a surface can be found by having atracker follow the 3D coordinates of an SMR moved over the surface. Itis possible to place a glass window on the top of the SMR to preventdust or dirt from contaminating the glass surfaces.

A gimbal mechanism within the laser tracker may be used to direct alaser beam from the tracker to the SMR. Part of the light retroreflectedby the SMR enters the laser tracker and passes onto a position detector.The position of the light that hits the position detector is used by atracker control system to adjust the rotation angles of the mechanicalazimuth and zenith axes of the laser tracker to keep the laser beamcentered on the SMR. In this way, the tracker is able to follow (track)the SMR.

Angular encoders attached to the mechanical azimuth and zenith axes ofthe tracker may measure the angle of rotation about the azimuth andzenith axes of the laser beam (with respect to the tracker frame ofreference). The one distance measurement and two angle measurementsperformed by the laser tracker are sufficient to completely specify thethree-dimensional location of the SMR.

As mentioned previously, two types of distance meters may be found inlaser trackers: interferometers and absolute distance meters (ADMs). Inthe laser tracker, an interferometer (if present) may determine thedistance from a starting point to a finishing point by counting thenumber of increments of known length (usually the half-wavelength of thelaser light) that pass as a retroreflector target is moved between thetwo points. If the beam is broken during the measurement, the number ofcounts cannot be accurately known, causing the distance information tobe lost. By comparison, the ADM in a laser tracker determines theabsolute distance to a retroreflector target without regard to beambreaks, which also allows switching between targets. Because of this,the ADM is said to be capable of “point-and-shoot” measurement.Initially, absolute distance meters were only able to measure stationarytargets and for this reason were always used together with aninterferometer. However, some modern absolute distance meters can makerapid measurements, thereby eliminating the need for an interferometer.

Some laser trackers include one or more cameras. A camera axis may becoaxial with the measurement beam or offset from the measurement beam bya fixed distance or angle. A camera may be used to provide a wide fieldof view to locate retroreflectors. A modulated light source placed nearthe camera optical axis may illuminate retroreflectors, thereby makingthem easier to identify. In this case, the retroreflectors flash inphase with the illumination, whereas background objects do not. Oneapplication for such a camera is to detect multiple retroreflectors inthe field of view and measure each in an automated sequence.

Some laser trackers have the ability to measure with six degrees offreedom (DOF), which may include three coordinates, such as x, y, and z,and three rotations, such as pitch, roll, and yaw. Several systems basedon laser trackers are available or have been proposed for measuring sixdegrees of freedom.

Laser scanners determine the 3D coordinates of points on an objectsurface by projecting a beam of light directly onto the surface and thencollecting and analyzing the reflected light. Laser scanners aretypically used for scanning closed or open spaces such as interior areasof buildings, industrial installations and tunnels. Laser scanners areused for many purposes, including industrial applications and accidentreconstruction applications. A laser scanner can be used to opticallyscan and measure objects in a volume around the scanner through theacquisition of surface points representing objects within the volume.

Some contemporary laser scanners may also include a camera mounted on orintegrated into the laser scanner for gathering camera digital images ofthe environment and for presenting the camera digital images to anoperator. By viewing the camera images, the operator can determine theextent of the measured volume and adjust the settings of the laserscanner to measure over a larger or smaller region of space. Inaddition, the camera digital images may be transmitted to a processor toadd color to the scanner image.

The acquisition of three-dimensional coordinates of surface points bylaser scanners may result in a large volume of data involving millionsof surface points. Many of these surface points may not be needed inorder to adequately represent objects or surfaces within the scannedvolume. Some extraneous data may be removed during postprocessing.

It is often the case that it is useful to characterize in threedimensions the environment surrounding the 3D measuring instrument.Ordinarily such information must be provided by an operator.Accordingly, while existing 3D measuring instruments are suitable fortheir intended purposes the need for improvement remains.

SUMMARY

In accordance with an embodiment of the invention, a coordinatemeasurement device is provided. The measurement device sends a firstbeam of light to a remote target, the remote target returning a part ofthe first beam of light as a second beam of light, the coordinatemeasurement device having a device frame of reference. The measurementdevice having a first motor and a second motor that cooperate to directthe first beam of light to a first direction. The first direction beingdetermined by a first angle of rotation about a first axis and a secondangle of rotation about a second axis, the first angle of rotationproduced by the first motor and the second angle of rotation produced bythe second motor. A first angle measuring device is provided thatmeasures the first angle of rotation and a second angle measuring devicethat measures the second angle of rotation. A distance meter is providedthat measures a first distance from the coordinate measurement device tothe remote target based at least in part on the second beam of lightreceived by an optical detector. A first portion of the coordinatemeasurement device rotates about the first axis. A second portion of thecoordinate measurement device rotates about the second axis. A thirdportion of the coordinate measurement device is fixed relative to themovements about the first axis and the second axis. A 3D time-of-flight(TOF) camera is positioned on a periphery of a portion of the coordinatemeasurement device, the portion selected from the group consisting ofthe first portion, the second portion, and the third portion, the cameraconfigured to acquire a camera image of an object. A processor isconfigured to determine at least one first three-dimensional (3D)coordinate in the device frame of reference of the remote target, the atleast one first 3D coordinate based at least in part on the firstdistance, the first angle of rotation, and the second angle of rotation,the processor further being configured to determine a plurality ofsecond 3D coordinates in the device frame of reference of the object,the plurality of second 3D coordinates being based at least in part onthe camera image, the first angle of rotation, and the second angle ofrotation.

In accordance with an embodiment of the invention, a coordinatemeasurement device is provided. The measurement device sends a firstbeam of light to a remote target, the remote target returning a part ofthe first beam of light as a second beam of light, the device having adevice frame of reference. The measurement device having a first motorand a second motor that cooperate to direct the first beam of light to afirst direction. The first direction determined by a first angle ofrotation about a first axis and a second angle of rotation about asecond axis, the first angle of rotation produced by the first motor andthe second angle of rotation produced by the second motor. A first anglemeasuring device is provided that measures the first angle of rotationand a second angle measuring device that measures the second angle ofrotation. A distance meter is provided that measures a first distancefrom the coordinate measurement device to the remote target based atleast in part on a first part of the second beam of light received by afirst optical detector. A light-field camera is positioned on thecoordinate measurement device, the light-field camera including amicrolens array and a photosensitive array, the light-field cameraconfigured to acquire a camera image of an object. A processor isconfigured to determine a first three-dimensional (3D) coordinate of theremote target in the device frame of reference, the 3D coordinate basedat least in part on the first distance, the first angle of rotation, andthe second angle of rotation. The processor further being configured tobring the object into focus and to determine a plurality of second 3Dcoordinates of the object in the device frame of reference, theplurality of second 3D coordinates based at least in part on the cameraimage, the first angle of rotation and the second angle of rotation.

In accordance with another embodiment of the invention, a method foroptically scanning and measuring an environment is provided. The methodincluding the steps of: providing a device having a first motor and asecond motor that cooperate to direct a first beam of light to a firstdirection, the first direction determined by a first angle of rotationabout a first axis and a second angle of rotation about a second axis,the device further having a first portion that rotates about the firstaxis, a second portion that rotates about the second axis and a thirdportion that is fixed relative to the movements about the first axis andthe second axis, the device further having a distance meter arranged toreceive a second light beam reflected by a remote target, the secondlight beam being a portion of the first light beam, the device having adevice frame of reference; providing a three-dimensional (3D) cameraoperably coupled to a periphery of one of the first portion, the secondportion or the third portion, the 3D camera selected from the groupconsisting of a time-of-flight (TOF) camera and a light-field camera;acquiring a camera image of an object with the 3D camera; rotating thefirst portion with the first motor to the first angle of rotation andthe second portion with the second motor to the second angle ofrotation; emitting the first beam of light and receiving the secondlight beam reflected off of the remote target; determining a firstdistance to the remote target with the distance meter in response toreceiving the second light beam; determining at least one first 3Dcoordinate of the remote target in the device frame of reference, the atleast one first 3D coordinates based at least in part on the firstdistance, the first angle of rotation and the second angle of rotation;and determining a plurality of second 3D coordinates of the object inthe device frame of reference, the second 3D coordinates based at leastin part on the camera image, the first angle of rotation and the secondangle of rotation.

BRIEF DESCRIPTION OF THE DRAWINGS

The subject matter, which is regarded as the invention, is particularlypointed out and distinctly claimed in the claims at the conclusion ofthe specification. The foregoing and other features and advantages ofthe invention are apparent from the following detailed description takenin conjunction with the accompanying drawings in which:

FIGS. 1A-1C show perspective views of exemplary laser trackers;

FIG. 2 shows computing and power supply elements attached to the lasertracker of FIG. 1;

FIG. 3 is a block diagram an electronics processing system associatedwith the laser tracker of FIG. 1;

FIG. 4 is a perspective view of a laser scanner device in accordancewith an embodiment of the invention;

FIG. 5 is a schematic illustration of the laser scanner of FIG. 4; and

FIG. 6 is a perspective view of the laser tracker of FIG. 1 configuredto respond to gestures from the operator.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Referring to FIG. 1, an exemplary measurement device, such as lasertracker 10 for example, is shown. An exemplary gimbaled beam-steeringmechanism 12 of laser tracker 10 comprises zenith carriage 14 mounted onazimuth base 16 and rotated about azimuth axis 20. Payload 15 is mountedon zenith carriage 14 and rotated about zenith axis 18. Zenithmechanical rotation axis 18 and azimuth mechanical rotation axis 20intersect orthogonally, internally to laser tracker 10, at gimbal point22, which is typically the origin for distance measurements. The lasertracker uses a beam of light, such as laser beam 46 for example, thatvirtually passes through gimbal point 22 and is pointed orthogonal tozenith axis 18. In other words, laser beam 46 is in the plane normal tozenith axis 18. Laser beam 46 is pointed in the desired direction bymotors within the tracker laser (not shown) that rotate payload 15 aboutzenith axis 18 and azimuth axis 20. Zenith and azimuth angular encoders,internal to the laser tracker (not shown), are coupled to zenithmechanical axis 18 and azimuth mechanical axis 20 and indicate, to highaccuracy, the angles of rotation. Laser beam 46 travels to externalretroreflector 26 such as a spherically mounted retroreflector (SMR) 26.By measuring the radial distance between gimbal point 22 andretroreflector 26 and the rotation angles about the zenith and azimuthaxes 18, 20, the position of retroreflector 26 is found within thespherical coordinate system of the laser tracker 10 (i.e. the deviceframe of reference).

Laser beam 46 may comprise one or more laser wavelengths. For clarityand simplicity, a steering mechanism of the sort shown in FIG. 1 isassumed in the following discussion and the claimed invention should notbe so limited. In other embodiments different types of steeringmechanisms are possible. For example, it would be possible to reflect alaser beam off a mirror rotated about the azimuth and zenith axes. Thetechniques described here are applicable, regardless of the type ofsteering mechanism.

In exemplary laser tracker 10, cameras 52 and light sources 54 arelocated on payload 15. Light sources 54 illuminate one or moreretroreflector targets 26. Light sources 54 may be LEDs electricallydriven to repetitively emit beams of pulsed light. Each camera 52comprises an optical detector, such as a photosensitive array forexample, and a lens placed in front of the photosensitive array. Thephotosensitive array may be a CMOS or CCD array. The lens may have arelatively wide field of view, say thirty or forty degrees. The purposeof the lens is to form an image on the photosensitive array of objectswithin the field of view of the lens. Each light source 54 is placednear camera 52 so that light from light source 54 is reflected off eachretroreflector target 26 onto camera 52. In this way, retroreflectorimages are readily distinguished from the background on thephotosensitive array as their image spots are brighter than backgroundobjects and are pulsed. There may be two cameras 52 and two lightsources 54 placed about the line of laser beam 46. By using two camerasin this way, the principle of triangulation can be used to find thethree-dimensional coordinates of any SMR within the field of view of thecamera. In addition, the three-dimensional coordinates of the SMR can bemonitored as the SMR is moved from point to point. A use of two camerasfor this purpose is described in commonly owned U.S. Published PatentApplication No. 2010/0128259 to Bridges which is incorporated byreference herein.

Other arrangements of one or more cameras and light sources arepossible. For example, a light source and camera can be coaxial ornearly coaxial with the laser beams emitted by the tracker. In thiscase, it may be necessary to use optical filtering or similar methods toavoid saturating the photosensitive array of the camera with the laserbeam from the tracker.

In the exemplary embodiment, the laser tracker 10 further includes athree-dimensional (3D) camera device 55. The 3D-camera device 55 iscapable of capturing both visual and distance information. As usedherein, a 3D-camera is a device having a single photosensitive arraycapable of determining the distance to an object surface over aplurality of pixels on the 3D-camera image sensor. Each of the pixelsalso corresponds to an angle relative to the 3D-camera image sensor.Both the distance and angle corresponding to each pixel can betransformed into the device frame of reference (i.e. the localcoordinate frame) of the laser tracker 10 using mathematical methodsthat are well known to users of ordinary skill in the art. Depending onthe type of 3D-camera image sensor, the 3D-camera image sensor may useusing either natural light or an external light source to obtain 3Dcoordinates. Unlike a scanner that uses triangulation principles with a3D-camera there may be no fixed geometrical relationship between the3D-camera image sensor and the light source. It should be appreciatedthat in most cases the accuracy of the 3D-camera is significantly lessthan that of the laser tracker 10. The 3D-camera may include, but is notlimited to a light-field camera and a time-of-flight (TOF) camera.

A light-field camera, sometimes referred to as a plenoptic camera,includes 3D camera that uses a microlens array to capture 4D light fieldinformation about the acquired image. In an embodiment, an array ofmicrolenses is placed at the focal plane of a camera main lens. An imagesensor is positioned slightly behind the microlenses. The image sensormight be a photosensitive array such as a CMOS or CCD array. Usingimages collected by the image sensor, the displacement of image partsthat are not in focus are analyzed to extract depth information. A lightfield camera may operate based on natural light, a light source coupledto the 3D camera or a light source external (decoupled) from the3D-camera.

One type of TOF camera uses an RF modulated light source with a phasedetector. Such TOF cameras are made by PMD Technologies GmbH of Siegen,Germany and Mesa Imaging AG of Zurich, Switzerland for example. Thesedevices work by modulating the outgoing beam with an RF carrier,measuring the phase shift of the reflected light, and determining adistance to the target based on the phase shift and on the speed oflight in air.

Another type of TOF camera is a range gated imager. Such TOF cameras aremade by Frunhofer IMS of Duisburg, Germany, and TriDiCam GmbH ofDuisbug, Germany for example. Range gated imagers include a built-inshutter in front of an image sensor. The shutter sends out light pulsesat the same rate the shutter opens and closes. By looking at thefraction of the light pulse received, the distance to the target iscalculated.

A third type of TOF camera is a direct TOF imager. Such a TOF cameramade by Advanced Scientific Concepts, Inc. of Santa Barbara, Calif. forexample, makes a variety of products referred to as 3D flash LIDARcameras. These devices emit a single pulse of laser light that reflectsoff objects before returning to camera, which includes a lens andphotosensitive array. The devices use a readout integrated circuit(ROIC) in a “trigger mode” to capture spatial and temporal informationusing a single pulse.

Many types of TOF 3D-cameras are possible, and the present invention isnot limited to the types described hereinabove. Each of the sensorarrays that provide distance measurements also provides anglemeasurements, as the angle to each point of an array of the TOF3D-camera may be calculated based at least in part on the position ofthe pixel and on the focal length of a lens within the 3D-camera.

In an embodiment, a 3D camera is positioned on a periphery of anexternal frame or surface of the laser tracker housing so as to enablerelatively large fields of view (FOV) to be obtained. In a contrastingmethod in which a 3D camera is located internal to a tracker, totalstation, or scanner. When located internal to the device, the FOV isnecessarily made very narrow by the presence of the exit aperture of the3D instrument. This limited FOV is avoided in an embodiment bypositioning the 3D camera 55 on the payload 15, which is the portion ofthe tracker that rotates about the axis 18. The FOV may be selected inthis case to be between 30 to 40 degrees so as to enable a work area tobe visualized in a single shot. However, because the payload may beturned about the axes 18 and 20 in FIG. 1A, the entire measurementvolume is accessible to viewing by the 3D camera 55. In an embodiment,the FOV of the 3D camera 55 located, when located on the payload 15, isat least +/−20 degrees (a full angle of 40 degrees).

In another embodiment shown in FIG. 1B, the 3D-camera 55 is disposed onthe zenith carriage 14, which is the portion of the tracker (or scanner)that rotates about the azimuth axis 20. In this case, the 3D-camera 55rotates about the azimuth axis 20 but remains at a fixed locationrelative to the zenith axis 18. Such a camera has a relatively wide FOVfor it to view a desired portion of the potential measurement volume ofthe tracker or scanner. In an embodiment the 3D camera 55 attached tothe zenith carriage 14 has a FOV of at least +/−40 degrees (a full angleof 80 degrees).

In an embodiment shown in FIG. 1C, a 3D-camera 55, fixed relative toboth the azimuth axis 20 and the zenith axis 18, is coupled to the base16. In an embodiment, since the 3D-camera 55 is fixed relative to themoving portion of the laser tracker or scanner, the 3D camera 55attached to the base 16 has a FOV of at least +/−60 degrees (a fullangle of 120 degrees).

In further embodiments represented by FIG. 1A, 1B, or 1C, the 3D camerais a light-field camera that provides not only 3D coordinates offoreground objects being measured but also has the ability to refocusand provide sharp images of the imaged objects after the image isacquired. This is even true if some of objects near to and far from thecamera are captured in the same image, as will often be the case whenthe 3D camera captures a wide FOV. As used herein, the term “focaldistance” means the in-focus distance from the 3D camera to a plane orpoint in which an object is positioned. Within a given image acquired bya light-field camera, there may be multiple objects or portions ofobjects, each at a different focal distance. In a light-field camera,each of these focal distances may be selectively determined duringsubsequent post-processing of the image. In one embodiment, all orsubstantially all of the objects within the image may be brought intofocus during post-processing of the image to provide an image where allof the objects are simultaneously in focus.

Although FIGS. 1A, 1B, and 1C show the 3D-camera 55 affixed to thetracker 10, it should be understood that a separate 3D-camera 55 may bedetached from the tracker. Such a separate 3D-camera may providereceived information to internal processors with the laser tracker 10 orto an external computer, such as computer 80 shown in FIG. 2.

Referring now to FIG. 2, an embodiment is shown of a laser tracker 10having an auxiliary unit 70. The auxiliary unit 70 supplies electricalpower to the laser tracker 10 and in some cases also provides computingand clocking capability. In one embodiment, the separate auxiliary unit70 is eliminated by moving the functionality of auxiliary unit 70 intothe tracker base 16. In most cases, auxiliary unit 70 is attached togeneral purpose computer 80. Application software loaded onto generalpurpose computer 80 may provide application capabilities such as reverseengineering. It is also possible to eliminate general purpose computer80 by building its computing capability directly into laser tracker 10.In this case, a user interface, possibly providing keyboard and mousefunctionality is built into laser tracker 10. The connection betweenauxiliary unit 70 and computer 80 may be wireless, such as through Wi-Fior Bluetooth communications, for example, or be wired through a cable ofelectrical wires, such as a serial, coaxial or Ethernet cable forexample. Computer 80 may be connected to a network, and auxiliary unit70 may also be connected to a network. In one embodiment, theapplication software is operated in a distributed computing environment.It should be appreciated that the computer 80 may be directly coupled tothe auxiliary unit 70, or may be remote from the laser tracker 10 andconnected via a local or wide area network. Plural instruments, such asmultiple measurement instruments or actuators for example, may beconnected together, either through computer 80 or auxiliary unit 70.

The laser tracker 10 may be rotated on its side, rotated upside down, orplaced in an arbitrary orientation. In these situations, the termsazimuth axis and zenith axis have the same direction relative to thelaser tracker as the directions shown in FIG. 1 regardless of theorientation of the laser tracker 10.

In another embodiment, the payload 15 is replaced by a mirror thatrotates about the azimuth axis 20 and the zenith axis 18. A laser beamis directed upward and strikes the mirror, from which it launches towarda retroreflector 26. In still another embodiment, the payload 15 may bereplaced by a two or more galvanometer mirrors that are rotatedindependently of each other to direct the laser beam to the desiredlocation.

The methods for operating the laser tracker 10 discussed herein may beimplemented by means of processing system 800 shown in FIG. 3.Processing system 800 comprises tracker processing unit 810 andoptionally computer 80. Processing unit 810 includes at least oneprocessor, which may be a microprocessor, digital signal processor(DSP), field programmable gate array (FPGA), or similar device.Processing capability is provided to process information and issuecommands to internal tracker processors. Such processors may includeposition detector processor 812, azimuth encoder processor 814, zenithencoder processor 816, indicator lights processor 818, ADM processor820, interferometer (IFM) processor 822, and color camera processor 824.As will be discussed in more detail below, the processing unit 810 mayalso include a 3D camera processor or engine 826. Auxiliary unitprocessor 870 optionally provides timing and microprocessor support forother processors within tracker processor unit 810. It may communicatewith other processors by means of device bus 830, which may transferinformation throughout the tracker by means of data packets, as is wellknown in the art. Computing capability may be distributed throughouttracker processing unit 810, with DSPs and FPGAs performing intermediatecalculations on data collected by tracker sensors. The results of theseintermediate calculations are returned to auxiliary unit processor 870.As explained herein, auxiliary unit 70 may be attached to the main bodyof laser tracker 10 through a cable, or it may be arranged within themain body of the laser tracker so that the tracker attaches directly(and optionally) to computer 80. Auxiliary unit 870 may be connected tocomputer 80 by connection 840, which may be an Ethernet cable orwireless connection, for example. Auxiliary unit 870 and computer 80 maybe connected to the network through connections 842, 844, which may beEthernet cables or wireless connections, for example.

It should be appreciated that while embodiments herein describe the useof the 3D-camera 55 with the laser tracker 10, this is for exemplarypurposes and the claimed invention should not be so limited. In oneembodiment shown in reference to FIGS. 4 and 5, the 3D-camera is usedwith a time-of-light (TOF) laser scanner 200. The term TOF is understoodto mean a measurement made based on a travel time of light in travelingbetween two points. As such, this method uses knowledge of the speed oflight of the air through which light travels. In a TOF of flight device,any method based on the travel time of light may be used to measure thetravel time between two points. For example, the light may be pulsed,and the time determined according to the travel time between a pulsewhen emitted and the pulse when returned. As another example, the lightmay be modulated in optical power in a sinusoidal pattern, and the timedetermine according to the travel time as calculated from a phase shiftin the sinusoidal modulation obtained from an optical detector at theTOF flight device. The TOF laser scanner 200 may be similar to the onedescribed in commonly-owned U.S. patent application Ser. No. 13/510,020filed on Nov. 11, 2010, the contents of which is incorporated herein byreference. In this embodiment, the laser light is emitted from the lightsource 202 and reflected off of a rotating mirror 204. The TOF scanner200 is rotated about a first axis 206, while the mirror is rotated abouta second axis 208 to optically scan the environment. The first axis 206is orthogonal to the second axis 208. The light is reflected off asurface in the environment and a portion returns along the path of theemitted laser light and is once again reflected by the rotating mirror204, whereupon it is collimated by a lens 210 and reflected off mirror212 into light receiver 214. Using the time it takes the laser light tobe emitted from the laser scanner 200 and reflected and returned, thedistance from the laser tracker to the surface may be determined. Usingangular measurements of rotation about the first axis 206 and the secondaxis 208, the 3D coordinates for points on the surface may be determinedin the laser scanner frame of reference.

In one embodiment, shown in FIG. 4, the 3D-camera 55 is mounted to aperiphery surface of a structure or housing 216. In another embodiment,shown in FIG. 5, the 3D-camera 55 is positioned internally to thescanner housing 216. In this embodiment, the 3D-camera 55 may bearranged co-axially with the axis 208 such that the 3D-camera 55 obtainsimages reflected from the mirror 204. It should be appreciated that whenthe 3D-camera is fixedly coupled to the TOF laser scanner 200 in a knownposition relative to the local coordinate system of the TOF laserscanner 200, the coordinates of points acquired by the 3D-camera 55 maybe transformed into coordinates of the local coordinate system of theTOF laser scanner 200 (i.e. the device frame of reference).

The TOF laser scanner 200 may also include a graphical display 218 thatdisplays a user interface. The user interface may be a touch screen thatallows the operator to interact with and control the operation of thescanner 200. In one embodiment, the three-dimensional images captured bythe 3D-camera 55 may be displayed on the graphical display 218.

In an embodiment, a TOF 3D-camera is provided on a laser tracker or aTOF laser scanner. In most cases, the TOF 3D-camera will have a loweraccuracy than the laser tracker or the TOF laser scanner, but it may beconfigured to rapidly provide a 3D image over a wide field of view,thereby enabling the laser tracker or TOF laser scanner to take furtheractions as needed. In the case of a laser tracker, a TOF 3D-camera mayidentify the outline of an object that is to be inspected and thendirect the operator, for example, by projecting a laser beam from thetracker, to each of a series of steps in an inspection plan. A TOF3D-camera may be used according to a variety of methods to assist inmeasurements made with a laser tracker or TOF laser scanner. Suchmethods are not limited by the examples given above.

Referring now to FIG. 6, one exemplary embodiment is illustrated thatuses the three-dimensional camera 55 for determining gestures, orcommand motions by the operator 400. In the illustrated embodiment, thecoordinate measurement device is a laser tracker 10 having the 3D-camera55 mounted to the payload 15 such that that 3D-camera 55 rotates aboutthe azimuth axis and zenith axis. In the exemplary embodiment, theoperator 400 is located adjacent the remote target being measured. Asused herein, the operator 400 is adjacent the remote target when theoperator 400 is positioned within the field of view of the 3D-camera 55.

In one embodiment, the 3D-camera engine 826 of FIG. 3 is a gesturerecognition engine that assists in evaluating or parsing of gesturespatterns to determine the performed gesture from a plurality ofgestures. In one embodiment, the 3D-camera 55 and engine 826 generate a3D skeletal model of the operator 400 from a plurality of surfacesmeasured of the operator measured by the 3D-camera 55. This allows forthe interpretation of movements and or body positions, such as theposition or orientation of the operators hand 404, as commands to beexecuted by the laser tracker 10. The skeletal model may includeinformation, such as the position of joints on the operator andlocations of specific body portions (e.g. hand 404, the arm 405). In oneembodiment, the skeletal model identifies the location of differentparts of the operator, such as the arm, elbow, hand, fingers and theconnecting joints for example.

The gestures engine 826 may include a collection of gesture filters,each comprising information concerning a gesture that may be performedby the user as interpreted through the skeletal model. The data capturedby camera 55 in the form of the skeletal model and movements of theskeletal model may be compared to the gesture filters in the gestureengine 826 to identify when an operator (as represented by the skeletalmodel) has performed one or more gestures. The gestures may be performedby one or more of the operator's body parts, or the relative movement orposition of those parts to each other (i.e. spatial configuration).Those gestures may be associated with various controls of the lasertracker 10. In other words, there may be a rule of correspondencebetween each of a plurality of gestures and each of the plurality ofcommands or controls for the laser tracker 10. Thus, the processingsystem 800 may use the gesture engine 826 to interpret movements of theskeletal model and control an application based on body (e.g. hand)position or movements.

The gesture filters may be modular or interchangeable. In oneembodiment, the filter has a number of inputs, each having a type, and anumber outputs, each having a type. Inputs to the filter may compriseitems such as joint data about a user's joint position (e.g. anglesformed by the bones that meet at the joint), RGB color data, and therate of change of an aspect of the user. Outputs from the filter mayinclude parameters such as a confidence level that a particular gesturehas been made and the speed of motion of the gesture. Filters mayfurther include contextual parameters that allow for the recognition ofparticular gestures in response to previous actions. The gestures thatmay be interpreted by the gesture engine 826 based on three-dimensionaldata acquired by the 3D-camara 55 include those disclosed in commonlyowned U.S. patent application Ser. No. 14/264,420 filed on Apr. 29,2014, which is incorporated by reference herein in its entirety.

It should be appreciated that while the 3D-camera 55 and the gestureengine 826 are illustrated in use with the laser tracker 10, this is forexemplary purposes and the claimed invention should not be so limited.In other embodiments, the 3D-camera 55 and gesture engine 826 may beused with another coordinate measurement device, such as the laserscanner 200 for example.

In still another embodiment, the 3D-camera 55 is coupled to the lasertracker 10 and acquires a three-dimensional image of the environmentaround the laser tracker 10. This image is then used to identify thelocation of the operator and allow the laser tracker 10 to rotate thepayload 15 about the azimuth and zenith axis to allow rapid acquisitionof the retroreflector 26 with the laser beam 46.

In still other embodiments, the 3D-camera 55 may be used with a lasertracker 10 in an automated system where the 3D-camera is used toidentify components within the process. For example, the 3D-camera 55may capture an image of the process and the engine 826 is used toidentify a desired object, such as a robot end effector. Using thisinformation, the laser tracker 10 transforms this information into anazimuth angle and a zenith angle to allow the rotation of the payload 15to the desired location and the rapid acquisition of a retroreflectivetarget.

The technical effects and benefits of embodiments of the inventioninclude allowing a 3D measurement device to quickly acquirethree-dimensional information about the environment or an object beingmeasured. Further technical effects and benefits of embodiments of theinvention provide for the association of rapidly 3D coordinates with afirst measurement device with 3D coordinates made with a secondmeasurement device in the same frame of reference. Still furthertechnical effects and benefits of embodiments of the invention providefor the determining and the carrying out of operational commands on themeasurement device in response to the movement, body position orgestures performed by an operator adjacent the object or area beingmeasured.

While the invention has been described in detail in connection with onlya limited number of embodiments, it should be readily understood thatthe invention is not limited to such disclosed embodiments. Rather, theinvention can be modified to incorporate any number of variations,alterations, substitutions or equivalent arrangements not heretoforedescribed, but which are commensurate with the spirit and scope of theinvention. Additionally, while various embodiments of the invention havebeen described, it is to be understood that aspects of the invention mayinclude only some of the described embodiments. Accordingly, theinvention is not to be seen as limited by the foregoing description, butis only limited by the scope of the appended claims.

What is claimed is:
 1. A coordinate measurement device that sends afirst beam of light to a remote target, the remote target returning apart of the first beam of light as a second beam of light, thecoordinate measurement device having a device frame of reference, themeasurement device comprising: a first motor and a second motor thatcooperate to direct the first beam of light to a first direction, thefirst direction determined by a first angle of rotation about a firstaxis and a second angle of rotation about a second axis, the first angleof rotation produced by the first motor and the second angle of rotationproduced by the second motor; a first angle measuring device thatmeasures the first angle of rotation and a second angle measuring devicethat measures the second angle of rotation; a distance meter thatmeasures a first distance from the coordinate measurement device to theremote target based at least in part on the second beam of lightreceived by an optical detector; a first portion of the coordinatemeasurement device that rotates about the first axis; a second portionof the coordinate measurement device that rotates about the second axis;a third portion of the coordinate measurement device that is fixedrelative to the movements about the first axis and the second axis; a 3Dtime-of-flight (TOF) camera positioned on a periphery of a portion ofthe coordinate measurement device, the portion selected from the groupconsisting of the first portion, the second portion, and the thirdportion, the camera configured to acquire a camera image of an object;and a processor configured to determine at least one firstthree-dimensional (3D) coordinate in the device frame of reference ofthe remote target, the at least one first 3D coordinate based at leastin part on the first distance, the first angle of rotation, and thesecond angle of rotation, the processor further being configured todetermine a plurality of second 3D coordinates in the device frame ofreference of the object, the plurality of second 3D coordinates beingbased at least in part on the camera image, the first angle of rotation,and the second angle of rotation.
 2. The coordinate measurement deviceof claim 1 wherein the remote target is a retroreflector target, thecoordinate measurement device further comprising: a position detector, asecond part of the second beam of light passing onto the positiondetector, the position detector configured to produce a first signal inresponse to a position of the second part on the position detector; anda control system that sends a second signal to the first motor and athird signal to the second motor, the second signal and the third signalbased at least in part on the first signal, the control systemconfigured to adjust the first direction of the first beam of light tothe position in space of the retroreflector target.
 3. The coordinatemeasurement device of claim 1 wherein the remote target is a surface ofthe object.
 4. The coordinate measurement device of claim 1 wherein the3D TOF camera includes an RF modulated light source and a phasedetector.
 5. The coordinate measurement device of claim 1 wherein the 3DTOF camera is selected from the group consisting of a range gated imagerand a direct TOF imager.
 6. The coordinate measurement device of claim 1wherein the 3D TOF camera is positioned on a periphery of the firstportion and the camera has a field-of-view (FOV) of at least +/−20degrees.
 7. The coordinate measurement device of claim 1 wherein the 3DTOF camera is positioned on a periphery of the second portion and thecamera has a FOV of at least +/−40 degrees.
 8. The coordinatemeasurement device of claim 1 wherein the 3D TOF camera is positioned ona periphery of the third portion and the camera has a FOV of at least+/−60 degrees.
 9. A coordinate measurement device that sends a firstbeam of light to a remote target, the remote target returning a part ofthe first beam of light as a second beam of light, the device having adevice frame of reference, the measurement device comprising: a firstmotor and a second motor that cooperate to direct the first beam oflight to a first direction, the first direction determined by a firstangle of rotation about a first axis and a second angle of rotationabout a second axis, the first angle of rotation produced by the firstmotor and the second angle of rotation produced by the second motor; afirst angle measuring device that measures the first angle of rotationand a second angle measuring device that measures the second angle ofrotation; a distance meter that measures a first distance from thecoordinate measurement device to the remote target based at least inpart on a first part of the second beam of light received by a firstoptical detector; a light-field camera positioned on the coordinatemeasurement device, the light-field camera including a microlens arrayand a photosensitive array, the light-field camera configured to acquirea camera image of an object; and a processor configured to determine afirst three-dimensional (3D) coordinate of the remote target in thedevice frame of reference, the 3D coordinate based at least in part onthe first distance, the first angle of rotation, and the second angle ofrotation, the processor further configured to bring the object intofocus and to determine a plurality of second 3D coordinates of theobject in the device frame of reference, the plurality of second 3Dcoordinates based at least in part on the camera image, the first angleof rotation and the second angle of rotation.
 10. A method for opticallyscanning and measuring an environment comprising: providing a devicehaving a first motor and a second motor that cooperate to direct a firstbeam of light to a first direction, the first direction determined by afirst angle of rotation about a first axis and a second angle ofrotation about a second axis, the device further having a first portionthat rotates about the first axis, a second portion that rotates aboutthe second axis and a third portion that is fixed relative to themovements about the first axis and the second axis, the device furtherhaving a distance meter arranged to receive a second light beamreflected by a remote target, the second light beam being a portion ofthe first light beam, the device having a device frame of reference;providing a three-dimensional (3D) camera operably coupled to aperiphery of one of the first portion, the second portion or the thirdportion, the 3D camera selected from the group consisting of atime-of-flight (TOF) camera and a light-field camera; acquiring a cameraimage of an object with the 3D camera; rotating the first portion withthe first motor to the first angle of rotation and the second portionwith the second motor to the second angle of rotation; emitting thefirst beam of light and receiving the second light beam reflected off ofthe remote target; determining a first distance to the remote targetwith the distance meter in response to receiving the second light beam;determining at least one first 3D coordinate of the remote target in thedevice frame of reference, the at least one first 3D coordinates basedat least in part on the first distance, the first angle of rotation andthe second angle of rotation; and determining a plurality of second 3Dcoordinates of the object in the device frame of reference, the second3D coordinates based at least in part on the camera image, the firstangle of rotation and the second angle of rotation.
 11. The method ofclaim 10 further comprising reflecting the first beam of light off of aretroreflector.
 12. The method of claim 10 wherein the device furtherincludes a rotating mirror.
 13. The method of claim 12 furthercomprising reflecting the first beam of light off of the rotating mirrorprior to the first beam of light striking the remote target andreflecting the second light beam off of the rotating mirror prior tobeing received by the distance meter.
 14. The method of claim 10 whereinthe 3D camera is positioned on a periphery of the first portion and the3D camera has a field-of-view (FOV) of at least +/−20 degrees.
 15. Themethod of claim 10 wherein the 3D camera is positioned on a periphery ofthe second portion and the camera has a FOV of at least +/−40 degrees.16. The method of claim 10 wherein the 3D camera is positioned on aperiphery of the third portion and the camera has a FOV of at least+/−60 degrees.
 17. The method of claim 10 wherein the object in thecamera image includes a plurality of surfaces, the plurality of surfacesincluding at least one surface.
 18. The method of claim 17 wherein theat least one surface is at least a portion of an operator's body, the atleast one surface including a first body part and a second body part.19. The method of claim 18 further comprising: providing a rule ofcorrespondence between each of a plurality of commands and each of aplurality of gestures, each gesture from among the plurality of gesturesincluding a spatial configuration of the first body part relative to thesecond body part, the rule of correspondence based at least in part onthe spatial configuration; performing a first gesture with the at leastone surface from among the plurality of gestures prior to capturing theimage, the first gesture corresponding to a first command; determiningthe first command based at least in part on the image according to therule of correspondence; and executing the first command with the device.20. The method of claim 19 wherein the 3D camera has a field of view,wherein the remote target is within the field of view when the firstgesture is performed.